1. Field of the Invention
The present invention relates to a method of decoding digital signals which correspond to images subdivided into fixed-size picture areas, called macroblocks, and have been transmitted and/or stored in the form of coded data corresponding to frames of I or P or B type. The I type being coded without using information from other frames, the P type being predicted by means of a motion-compensated prediction from an earlier reference frame and an associated motion information, and the B-type being bidirectionally interpolated from an earlier and a later reference frame of I or P type and associated motion information. Such decoding method includes for decoding of the macroblocks, the steps of:
storing the input bitstream of the coded macroblock data of I, P or B type, corresponding for each macroblock to its relevant parameters and the picture elements therein; PA1 decoding said coded data according to their type; and PA1 displaying pictures reconstituted in accordance with the decoded data thus obtained. PA1 means for storing the input bitstream of the coded macroblock data of I, P, or B type; PA1 means for decoding said coded macroblock data according to said type; PA1 means for storing at least said earlier reference frame; PA1 means for displaying after a format conversion the pictures reconstituted in accordance with the decoded data thus obtained; PA1 bus means for the connections between said storing, decoding and displaying means. PA1 (a) input buffer memory 110: about 2.5 Mbits; PA1 (b) reference memories for the reference pictures (it has been already said that these previous and following pictures are necessary for the bidirectional interpolation of macroblocks of B pictures): 720.times.576.times.8.times.1.5 Mbits per picture, i.e. 5 Mbits; PA1 (c) memory for each B picture: about 3.8 Mbits. PA1 (a) the intra picture I.sub.1 is decoded and stored within the first reference memory (FIG. 6) as long as another picture is not ready to replace it (predicted picture P.sub.7), i.e during six picture periods; PA1 (b) during the two following picture periods, the coded pictures B.sub.-1 and B.sub.0, which follow the picture I.sub.1, are bidirectionally interpolated and stored within the display memory (FIG. 8) in view of the display (FIG. 9) of each field (odd field+even field) of these pictures; PA1 (c) as for the picture I.sub.1, the predicted picture P.sub.4 is similarly decoded and stored within the second reference memory (FIG. 7) as long as another picture is not ready to replace it (predicted picture P.sub.10); PA1 (d) the picture I.sub.1 is displayed (odd and even fields); PA1 (e) the coded pictures B.sub.2 and B.sub.3 are interpolated and stored within the display memory (FIG. 8) in view of the display (FIG. 9) of both fields of these pictures; PA1 (f) the picture P.sub.4 is displayed, . . . and so on: the decoding and display steps are carried out, in the transmission order for the decoding one and in the original order for the display one. PA1 (1) a first sub-step at nominal speed for decoding among the fully coded data corresponding to the macroblock only the relevant macroblock parameters, the obtained decoded data being then stored in a part called "auxiliary memory" of said external memory; PA1 (2) a second sub-step for decoding among said fully coded data only data corresponding to the picture elements, said second sub-step being performed at the time decoded data are required at display side, according to said "decoding-on-the-fly" principle which excludes any storing step of the decoded data associated to the picture elements. PA1 (1) a first decoding sub-step for decoding among the coded macroblock data only the relevant macroblock parameters, the decoded signals thus obtained being stored as auxiliary data; PA1 (2) a second decoding sub-step for decoding among said coded macroblock data only the data corresponding to the picture elements, said second sub-step being performed at the speed decoded data are required for the displaying step, according to a "decoding-on-the-fly" principle excluding any storing step of said decoded data associated to the picture elements of the predicted or interpolated macroblocks; PA1 (3) a third decoding sub-step for performing, after the decoding operation of the later reference frame, the updating of said earlier reference frame, which is replaced by said later one. PA1 auxiliary means for storing the decoded data corresponding only to the relevant parameters of each macroblock; PA1 means for controlling said storing means, said bus means, and said displaying means so as to process the decoding of said coded macroblock data according to three successive sub-steps respectively provided for:
The invention also relates to a decoding device for the implementation of this of decoding digital signals which correspond to images subdivided into fixed-size picture areas, called macroblocks, and have been transmitted and/or stored in the form of coded data corresponding either to frames of I or B type, the I type being coded without using any information from other frames the P type being predicted by means of a motion-compensated prediction from an earlier reference frame and an associated motion information, the B type being bidirectionally interpolated from an earlier and a later reference frame of I or P type and associated motion information, said decoding device comprising:
Such an invention has an application in the field of digital television (either normal definition television or high definition television), and especially when television signals are compatible with the MPEG standard for digital moving picture decoding, the characteristics of which will not be described.
2. Description of the Related Art
The International Standardization Organisation (ISO) has developed over these last few years several digital data compression algorithms, and particularly the MPEG standard (MPEG, from Moving Picture Experts Group). This standard, provided for achieving compatibility between compression and decompression equipment and specifying both the coded digital representation of a video signal and the method of decoding, is based on two main techniques: an orthogonal transformation of the signals to be compressed, which allows to remove the spatial redundancy within each picture, and a motion compensation between pictures, which allows to remove the temporal redundancy in the sequence of pictures.
Motion compensation is a technique which consists in combining already coded pictures with data representative of the motion concerning these pictures. This technique, also called motion-compensated prediction, assumes that the current picture can be locally modeled as a translation of the picture at a previous time. The word "locally" means that the amplitude and the direction of the translation need not be the same everywhere in the picture. The pictures being subdivided into blocks, or into macroblocks including several blocks, the translation from one macroblock (for instance) of a previous picture to a corresponding macroblock of the current picture is described by a vector representative of the displacement between these macroblocks. The encoding process can then concern no longer the macroblock itself but an error macroblock obtained by subtraction between said previous and current macroblocks. An inverse decoding process allows to reconstruct the original pictures. Under the MPEG standard, the picture is divided into a series of slices, each of them containing a number of macroblocks. Each macroblock covers an area of 16.times.16 picture elements, or pixels, comprising itself four luminance blocks of 8.times.8 pixels and two chrominance blocks of 8.times.8 pixels, and the motion compensation is provided between the macroblocks of 16.times.16 pixels (by coding the motion of an entire macroblock by a motion vector rather than at the pixel level, an increased data compression is achieved).
Three types of pictures (designated "intra", "predicted", and "interpolated") are considered in the MPEG standard: I-type pictures, said intra pictures being coded without using any information from other pictures (these I pictures, least frequently provided, allow for instance random access to the sequence of coded pictures), P-type pictures, said predicted pictures being predicted by motion compensation from a previous I or P picture (only the incremental changes in pixel values from this last picture are coded), and B-type pictures, said interpolated pictures being obtained by a bidirectional motion-compensated interpolation from both an earlier and a later picture of I or P type.
The corresponding motion vector generation is the following. For the predicted pictures, one forward motion vector (making reference to a previous picture field) is calculated for each field macroblock and with respect to the preceding predicted or intra picture field of the same parity (i.e. the previous predicted odd, resp. even, picture field if the current picture field is odd, resp. even). For the bidirectionally interpolated pictures, forward and backwards motion vectors are calculated with respect to the preceding and the following predicted or intra picture field of the same parity (and practically a third displacement vector is often calculated with respect to the nearest predicted or intra picture field of the opposite parity).
U.S. Pat. No. 5,294,974 shows the structure of an encoding device provided for yielding a digital data stream with a syntax according to the MPEG standard, while a decoder provided for decoding such an MPEG digital data stream is described for example in the document ISCAS 94, London, May 1994, chap. 3.22, "VLSI implementation of MPEG decoders". The structure of a decoder as described in this document is shown in to FIG. 1. hereof Input coded signals are received by a decoding sub-assembly 10, the output signals of which correspond to the luminance (and the chrominance) of the pixels to be displayed. The sub-assembly 10, connected via a bus 30 to a storing sub-assembly 20, comprises a decoding device 11 and a display device 12. The decoded signals delivered by the decoding device 11 are stored in the storing sub-assembly 20 in the form of macroblocks of 16.times.16 pixels and according to the so-called 4:2:0 format. The stored macroblocks are then sent back towards the display device 12 which adapts them to the wanted output interlaced mode 4:2:2 (and also reconstitutes the lacking chrominance information).
FIG. 2 shows how the decoding device 11 functions according to the MPEG standard. The coded signals of the MPEG data stream are first stored in a buffer memory 110, in which the header of the picture sequence and of each group of pictures in this sequence will be read. The beginning of each picture and its type being identified, the decoding step of each macroblock is then implemented by a variable length decoding circuit 120. After an inverse quantization of each data block or macroblock followed by its inverse orthogonal transform (respectively in an inverse quantization circuit 130 and in a inverse discrete cosine transform circuit 140), a motion compensation circuit 150 receives the displacement vectors and the signals indicating the type of each macroblock (intra or inter, according to the coding mode) and yields the predicted macroblocks. Except with the intra mode, these macroblocks are then added to the decoded signals by an adder 160, the output of which delivers the signals that will be displayed in view of the reconstruction of the pictures. If these pictures are of I or P type, they are stored in reference memories since they will be useful for the reconstruction of B pictures. As the pictures had been transmitted and/or stored in a different order from the order in which they will be displayed (in view of this reconstruction of B pictures), these pictures must be reordered before they are displayed.
Most of MPEG decoders usually include an external memory of 16.8 Mbits (1 Mbit=2.sup.20 bits). For picture sizes of for instance 720.times.576 pixels, this memory capacity is subdivided as indicated below:
The global memory capacity thus has a value of 16.3 Mbits (2.5+2.times.5+3.8), and so only 500 kbits are therefore available for further applications (for instance additional functions like teletext, graphics, on-screen-display, and so on), which is not sufficient to implement them.
European patent application No. 94402708.5, first filed on, or based on a priority of, Nov. 25th, 1994, originating from the present applicant and not yet published, described a method of decoding MPEG signals that allows to correct this drawback and therefore to provide available memory capacity for such further applications. However, before this method is described, it is useful to recall in reference to FIGS. 3 to 9 the manner in which the buffer memory of an MPEG decoder is used.
FIG. 3 shows the input coded data stream of said decoder for a complete group of twelve pictures (GOP), said pictures being of I, P or B type (to use bidirectional prediction, the original order of the pictures, B.sub.-1 B.sub.0 I.sub.1 B.sub.2 B.sub.3 P.sub.4 B.sub.5 B.sub.6 P.sub.7 B.sub.8 B.sub.9 P.sub.10 B.sub.11 B.sub.12 has been modified and is become I.sub.1 B.sub.-1 B.sub.0 P.sub.4 B.sub.2 B.sub.3 P.sub.7 B.sub.5 B.sub.6 P.sub.10 B.sub.8 B.sub.9 I.sub.13, . . . , and so on). These input data are delivered to the buffer memory 110, here of FIFO type (First-In, First-Out), and FIG. 4 shows the evolution of the storing step in this FIFO memory. When all data concerning a given picture are stored within this memory, the MPEG decoder can decode the picture (FIG. 5), according to the following time schedule:
In comparison with such a conventional MPEG decoder, said European patent application has proposed to implement the decoding step of B pictures at the moment when decoded data corresponding to B pictures are requested in view of the display step, said B pictures being therefore decoded according to a "decoding-on-the-fly" principle (the corresponding decoded data are no longer stored within the display memory), and concerns to this end a method of decoding digital signals which correspond to images with interlaced fields and which, after a data compression based on a sequential coding of macroblocks partitioning said images and each including luminance and chrominance blocks, have been transmitted and/or stored in the form of coded data corresponding either to frames of I or B type, the I type being coded without using information from other picture frames, the P type being predicted by means of a motion-compensated prediction from an I-frame or another P-frame, the B-type being bidirectionally interpolated from previous and following frames of I or P tppe. This method uses a memory space in said external memory for storing the frames which have been or are being decoded and will then be displayed, and comprises, for the decoding step of at least luminance blocks of each macroblock of the B frames, two successive sub-steps:
Since no orthogonal transform is involved in the first sub-step, its implementation is simpler than a conventional variable length decoding step and, since all relevant macroblock parameters can now be retrieved from the auxiliary memory, this allows to have a very fast second sub-step. Such a two step decoding approach allows saving of around 3 Mbits in memory capacity.
Although it would be very attractive to achieve a further reduction of such a memory capacity, it seemed that it would not be possible without affecting the memory bandwidth significantly.